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ABSTRACT 



This report studies career academies to explore students ' 
and teachers' experiences and to contrast them with those of their peers in 
regular high school environments. Data are from questionnaires, interviews, 
and observations of Academy and non- Academy students (835 and 686 
respectively) and teachers (65 and 403 respectively) . Chapter 2 describes the 
data sources and research samples and presents analytical issues relevant to 
data interpretation. Chapter 3 examines the extent to which the career 
academy approach increases students' self-reported exposure to selected 
dimensions of school - related support from teachers, peers, parents, and other 
adults. It reports that students who experience higher levels of support are 
more likely to give intrinsic motivations for participating in school 
activities and see strong connections between what they are learning in 
school and their futures; increases in student motivation are related to 
higher levels of students' self-reported engagement. Chapter 4 examines the 
extent to which the career academies provide teachers with supports that 
enhance their job satisfaction and sense of effectiveness in making a 
difference in their students' lives. It explores whether the structural 
changes created by the career academy approach are associated with 
differences in teachers’ level of collaboration with each other, in their 
classroom resources and control over their work, and in their relationships 
with students. (Appendixes contain 57 references and data tables.) (YLB) 
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Preface 



The Career Academy approach stands today at the intersection of several prominent trends 
in education reform. Although they have existed for more than 25 years, Career Academies have 
gained greater recognition recently as states and school districts have tried to build on the 
momentum of the high school restructuring and school-to-work transition movements. Interest in 
Career Academies was also accelerated by the reauthorization of the Carl D. Perkins Vocational 
and Applied Technology Act in 1990 and the passage of the federal School-to-Work Opportunities 
Act in 1994. Consistent with the goals of Career Academies, both of these legislative initiatives aim 
to ensure that high school students graduate with skills they can apply in the labor market and that 
they have access to clearer pathways from high school to postsecondary education and work. 

Like a growing number of restructuring initiatives, Career Academies aim to support more 
constructive relationships between high school students and teachers. And, the Academies 
emphasis on providing students with a strong academic experience while preparing them for life 
beyond high school is consistent with several provisions in the Perkins Act designed to enhance 
teaching and learning strategies that integrate academic and occupation-related knowledge and 
skills. Finally, the Academies’ employer partnerships and work-based learning component are 
directly relevant to the School-to-Work Opportunities Act, which authorized federal funding for 
states and localities to help schools forge stronger relationships with their communities and local 
employers. 

In 1993, the Manpower Demonstration Research Corporation (MDRC) began an evaluation 
of the Career Academy approach in 10 sites around the country. Its primary goal is to provide 
policymakers and educators with definitive evidence about the impact Career Academies have on 
students’ success in high school and their transition to further education and the labor market. It 
will also offer lessons about how Career Academies operate and are sustained and about the 
pathways through which they affect student engagement and performance in school. 

The current report builds on the foundation laid in MDRC’s first report from the evaluation 
— Career Academies: Early Implementation Lessons from a 10-Site Evaluation. Whereas the first 
report described the structural elements of the Career Academy approach as they had been 
implemented and sustained in the 10 sites, the current report takes an early look inside the 
Academies to shed light on the experiences of their students and teachers and to contrast these 
experiences with those of their peers in the participating high schools. The findings from these two 
reports suggest that the Career Academy approach holds promise for restructuring high schools 
with the aim of creating more mutually supportive student and teacher “communities.” The first 
report offered useful lessons from the sites about what it takes to implement the approach and to 
adapt it to local needs and circumstances. The current report draws on information collected from 
teachers and from students, most of whom were in the 9th or 10th grade at the time of data 
collection. It offers important insights into the ways in which Career Academies enhance the 
institutional and interpersonal supports that are associated with strong teaching and learning 
communities. While these supports appear to have translated into at least modest increases in 
student motivation and higher job satisfaction among teachers, we do not yet see significant 
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enhancements in students engagement in school (but, in part, this is because the students in the 
sample were already fairly highly engaged in school). 

At this early stage in the evaluation, the initial evidence suggests that the Career Academy 
approach has the potential to improve high schools. However, it is still too early to tell whether the 
overall experience will have a significant impact on students’ performance in high school and on 
helping them move into postsecondary education and the labor market. At the time the data for this 
report were collected, very few of the students had participated in the Academies’ work-based 
learning activities and they had only limited exposure to the integrated academic/vocational 
curricula. Also, it remains to be seen whether the types of supports discussed in this report will have 
a cumulative effect on student engagement and performance as they continue over an additional 
two or three years. 

It will be critical to follow students further through their high school years and beyond. 
Future reports from the evaluation will examine students’ exposure to the full range of Career 
Academy components. They will also include longer follow-up periods to examine the potential 
cumulative effects the Academies may have on students’ progress through high school, graduation, 
and the transition to postsecondary education and work. 

It is our hope that this early look at 10 Career Academies will be useful to educators and 
policymakers who are involved in school reform efforts across the country. 



Judith M. Gueron 
President 
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Executive Summary 



In 1993, the Manpower Demonstration Research Corporation (MDRC) began a 10-year 
evaluation of the Career Academy approach — a widely established school reform initiative that 
encompasses the key principles of the national school-to-work (or school-to-career) movement. 
As such, the primary goals of the Career Academy approach are to improve students perform- 
ance in high school and to provide them with clearer pathways to post-secondary education and 
careers. While attempting to create more supportive teaching and learning communities within 
high schools, Career Academies also seek to integrate academic and vocational instruction and to 
provide work-based learning opportunities for students, with the aim of preparing them for their 
lives beyond high school — whether they are going straight into the job market or planning to 
attend college first. 

This report is the second in a series from MDRC’s Career Academies Evaluation, which 
focuses on 10 high schools and their Career Academies from across the country. The first report 
— Career Academies : Early Implementation Lessons from a 10-Site Evaluation described the 
10 Career Academies participating in the study and their local contexts. The current report begins 
to look inside the participating Career Academies and focuses on the extent to which they func- 
tion as “communities of support” for students and teachers. For students, such support includes 
the personalized attention they get from their teachers, their teachers’ expectations of them, their 
fellow classmates’ level of engagement in school, and the opportunities they have to collaborate 
with their peers on school projects. Teachers are supported by, among other things, opportunities 
for professional collaboration and development, adequate resources, the capacity to influence in- 
structional and administrative decisions, and opportunities to give personalized attention to stu- 
dents. Both this study and previous research have identified these dimensions of support as 
factors that can have an important effect on both students’ motivation and engagement in school 
and teachers’ job satisfaction and sense of whether they are making a difference in their students’ 
lives. 

The key findings reported here indicate that the Career Academies provide their students 
and teachers with a greater degree of institutional and interpersonal support than is available to 
fneir non-Academy counterparts in the same comprehensive high schools. Students in the early 
stages of their Academy experience report that they are somewhat more motivated to attend 
school and that their schoolwork seems more relevant to their future education and career goals. 
At the same time, while Academy students appear to be highly engaged in school, they do not 
appear to be more engaged than their non-Academy counterparts. Academy teachers were more 
likely to see themselves as belonging to a strong professional community and indicated higher 
levels of job satisfaction than their non-Academy counterparts in the same high schools. Never- 
theless, Academy and non-Academy teachers were about equally likely to rate themselves as 
being highly effective with their students. 

These early benchmarks of contrast between the Academy and non-Academy school en- 
vironments, as viewed through the eyes of their students and teachers, should be interpreted with 



caution. Most important, the current report focuses on only a limited set of student experiences 
that are likely to be affected by the Career Academies. Many of the students participating in the 
study had gone through only one high school year at the time the data for this report were col- 
lected, and only about one-third of the students had reached their second year in the study. Thus, 
they had little or no exposure to some of the key Academy components, particularly its integrated 
academic/occupational curricula and work-based learning opportunities. Future reports will ex- 
amine a broader set of indicators of student performance and engagement in school and at work, 
and will capture the cumulative effects, if any, as they accrue through high school and beyond. 



What Is a Career Academy ? 

Career Academies are organized as "schools- with in- schools” in which groups of students 
(usually 30 to 60 per grade in grades 9 through 12 or 10 through 12) take several classes together 
each year with the same group of teachers. The goal of the school- within-a-school is to promote 
more constructive relationships between and among teachers and students and thereby to increase 
students engagement and success in high school. Each Academy focuses on a career theme, such 
as business and finance, electronics, or health occupations, to provide opportunities for teachers 
and students to integrate academic and occupation-related classes in an effort to enhance their 
relevance to the real world while preserving academic rigor. Academies also forge partnerships 
with local employers from a particular field to help plan and guide the program, and to serve as a 
source of adult mentors and work internships for the students. 

While Career Academies have existed for over 25 years, the approach has gained greater 
prominence recently as states and school districts have increased their efforts to restructure high 
schools. This restructuring is aimed at supporting students academically while providing them 
with marketable skills, work-based learning experiences, and smoother transitions to post- 
secondary education and productive employment. Interest in Career Academies was further ac- 
celerated with the passage of the federal School-to-Work Opportunities Act in 1994. The Act 
provided federal funding and support for states and localities to take a systemic approach to 
helping schools forge stronger partnerships with their communities and with local employers, 
and to create opportunities for students to begin making connections between schooling and their 
career aspirations. The core components of the Career Academy approach reflect many of the 
cornerstones of the new legislation and its objectives as well as many key dimensions of other 
reform efforts to improve high schools. While there are no reliable data on the total number of 
Career Academy programs nationally, current estimates suggest that Career Academies have 
been established in at least 600 to 700 high schools. 



The Ca reer Academies Evaluation 

Ten high schools and their Career Academies, representing most of the major, established 
networks of Career Academies across the country, are participating in this study. Their names, 
locations, and affiliations are shown in Exhibit 1. Most of the nine school districts in the 



Exhibit 1 

Career Academies Evaluation 

Names, Locations, and Affiliations of Participating Career Academies 




2 . 



Business and Finance Academy 
George Westinghouse High School 

Academy of Finance 

Lake Clifton/Eastern High School 



3. Public Service Academy 
Anacostia High School 

4 , Academy for Aerospace Technology 
Cocoa High School 



Pittsburgh Public Schools 
Pittsburgh, Pennsylvania 

Baltimore City Public Schools 
Baltimore, Maryland 

District of Columbia Public Schools 
Washington, D.C. 

Brevard County Public Schools 
Cocoa, Florida 



Independent 
1984-85 

National Academy Foundation 
1987-88 

D.C. Public Schools Academy Network 
1989-90 

Florida's Academies for Career Development 

and Applied Technology 

1993-94 



5. 


Academy of Travel and Tourism 
Miami Beach Senior High School 


Dade County Public Schools 
Miami Beach, Florida 


National Academy Foundation 
1991-92 


6. 


Health Professions Academy 
Socorro High School 


Socorro Independent School District 
Socorro, Texas 


Independent 

1991-92 


7. 


Global Business Academy 
Valley High School 


Santa Ana Unified School District 
Santa Ana, California 


California Partnership Academy 
1991-92 


8. 


Watsonville Video Academy 
Watsonville High School 


Pajaro Valley Unified School District 
Watsonville, California 


California Partnership Academy 
1991-92 


9. 


Electronics Academy (SC) 
Silver Creek High School 


East Side Union High School District 
San Jose, California 


California Partnership Academy 
1984-85 


10. 


Electronics Academy (I) 
Independence High School 


East Side Union High School District 
San Jose, California 


California Partnership Academy 
1984-85 



evaluation (one district includes two of the participating Career Academies) are in urban areas or 
small cities and enroll substantial percentages of black and Hispanic students compared with na- 
tional averages. The participating school districts also have, on average, higher dropout rates, 
unemployment rates, and percentages of low-income families. Most Career Academies across the 
country are located in such districts, and MDRC purposely sought such sites for the Career 
Academies Evaluation. Each of the participating Career Academies attempted to serve a wide 
range of students, including those who were doing well in school and those who appeared to be 
at risk of dropping out. 

As they entered the study, each of the 10 sites had already established the basic Career 
Academy components mentioned above: a school-within-a-school organization, an integrated 
academic/occupational curriculum, and partnerships with local employers. This combination of 
features was not available elsewhere in the participating high schools. Exhibit 1 indicates that the 
Career Academies in the study reflect a range of occupational themes: three are in the business 
and finance fields, three focus on high-technology areas such as electronics and aerospace tech- 
nology; and one each is in the fields of health occupations, public service, travel and tourism, and 
video technology. 

The Career Academies Evaluation will follow nearly 2,000 students from the 10 sites 
through their high school years and for up to three years following their scheduled graduation 
from high school. Each of these students was identified by the participating Career Academies as 
eligible for participation in their programs. Because each of the programs received applications 
from more students than they could serve, however, approximately 1,100 of these students were 
randomly selected to enroll in the programs; the remaining students were not selected and chose, 
or were assigned, to attend other regular high school programs. This random assignment research 
design provides a unique foundation for an unusually rigorous assessment of the Academies’ ef- 
fects on student outcomes. 



This report draws on students’ and teachers’ responses to questionnaires they completed 
during their first or second year in the study. From the full sample, 1,406 students and 468 teach- 
ers in the Academy and non- Academy environments of the 10 high schools completed a ques- 
tionnaire. They were asked a range of questions about their experiences, behaviors, and attitudes 
in school, and their responses were then used to create indicators of factors that were likely to 
affect the students’ engagement and performance. Most of the analyses conducted for this report 
focus on the proportion of students or teachers who gave consistently “high” ratings across the 
groups of questionnaire items. In general, a “high” rating indicates that a given respondent con- 
sistently and strongly affirmed a group of statements about his or her experiences, behaviors, and 
attitudes. The report also draws on qualitative information collected during on-site interviews 
with teachers and students and observations of Academy classes and activities. 

It is important to note that there were no systematic differences in the background char- 
acteristics of the Academy and non-Academy students who completed the questionnaire. This 
increases the confidence one may have that any differences that emerge based on the question- 
naire data can be attributed to the difference between the Academy and non-Academy environ- 
ments. Nonetheless, the results should be interpreted somewhat cautiously because there may be 



some unmeasured differences between the background characteristics of Academy and non- 
Academy students who completed a questionnaire. Also, there were some modest differences 
between the characteristics of the students who completed a questionnaire and those students 
who did not. 

Academy and non- Academy teachers who completed the questionnaire had several simi- 
lar key background characteristics. However, some measured differences emerged for other char- 
acteristics, for which statistical controls were applied in the analysis. Thus, while differences 
between Academy and non- Academy teachers that emerge from the questionnaire data may be 
partly attributable to the Academies, they may also be the result of some underlying differences 
in the unmeasured characteristics of the two groups of teachers. In general, the findings from the 
comparison of Academy and non- Academy teachers is intended to highlight differences in the 
teaching and learning opportunities being made available to Academy and non-Academy stu- 
dents. 

The Career Academies Evaluation is being supported by the U.S. Departments of Educa- 
tion and Labor, 15 private funders, and the 10 sites participating in the study. 



Career Academies as Communities of Support 

Where the first report from the evaluation was concerned with the structural elements of 
the Career Academy approach as they had been implemented and sustained in the 10 sites, the 
current report takes an early look inside the Academies to shed light on the experiences of their 
students and teachers and to contrast these experiences with those of their peers in the partici- 
pating high schools. In particular, the report focuses on whether the Career Academies serve as 
“communities of support” for students and teachers. In the context of this report, such communi- 
ties provide a range of institutional and interpersonal supports that aim to enhance student moti- 
vation and engagement in school and to help teachers optimize their students’ learning 
experiences. 

Exhibit 2 shows two simplified conceptual models of how selected institutional and in- 
terpersonal supports are hypothesized to affect student engagement and teacher effectiveness, 
respectively. For students, the Career Academy approach (particularly its school-within-a-school 
organization) is hypothesized to enhance support from teachers, peers, and parents which will 
help motivate them to do well and lead to greater engagement. Similarly, the Career Academy 
approach is hypothesized to offer teachers institutional supports (such as teacher collaboration, 
resources, and ability to influence work), which enhance certain interpersonal supports and, ulti- 
mately, increase their job satisfaction and sense of effectiveness. Measures of the constructs in 
Exhibit 2 were created from the groups of student and teacher questionnaire items described 
above. 

All of the differences between Academy and non-Academy participants discussed below 
are statistically significant, except when noted otherwise. Statistically significant differences are 
those that are least likely to be due to chance. 
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Exhibit 2 



Career Academies Evaluation 

Relationships Among Constructs That Define the Career Academy 
"Communities of Support" for Students and Teachers 
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How Do Career Academies Support Students? 

• Career Academies increased, at least modestly, the support students re- 
ceive from their teachers and peers. 

The first set of bars in Exhibit 3 indicates that Academy students were more likely than 
their non- Academy counterparts to report that their teachers give them personalized attention and 
have high expectations of them. The second set of bars shows that Academy students were also 
more likely to see their classmates as being engaged in school and to work with them on school 
projects and assignments. In general, these findings suggest that the structural features of the Ca- 
reer Academy approach — particularly, in this case, the school-within-a-school organization 
offgr students a greater degree of support from teachers and peers than is available to similar stu 
dents in the regular high school environments. 

• These supports appear to have enhanced student motivation and in- 
creased the connections they see between what they are learning in school 
and their longer-term education and career interests. 

The third set of bars in Exhibit 3 indicates that Academy students were slightly more 
likely than non-Academy students to indicate that they were intrinsically motivated to attend 
school. That is, these students reported that they attend school primarily because they like it and 
are interested in what they are learning, rather than because they must or primarily in response to 
external pressures or the negative consequences of not attending. The fourth set of bars indicates 
a further difference between Academy and non-Academy students: Academy students were more 
likely to perceive a strong connection between what they are learning in school and their longer- 
term education and career interests. 

• Although most Academy students indicated that they were highly en- 
gaged in school, they were no more likely to do so than their non- 
Academy peers. 

In this analysis, “engagement in school” includes students self-reported behavior (such 
as being prepared for and paying attention in class, exerting effort in class, and doing home- 
work), emotional state when in school, and psychological commitment to doing well in school. 
Further analyses of the questionnaire showed that, for both Academy and non-Academy students, 
both the interpersonal supports and the motivational processes were related to this self-reported 
measure of school engagement. Thus, the Academy students’ higher levels of support, motiva- 
tion, and belief that their schoolwork has future relevance should translate into higher levels of 
engagement in school. However, as indicated in Exhibit 3, the questionnaire data indicate that, at 
this early point in the follow-up period, the Academy and non-Academy students reported similar 
levels of behavioral, psychological, and emotional engagement. In all, about two-thirds of both 
Academy and non-Academy students were highly engaged in school, indicating that students 
were likely to be engaged in school at this point in their high school careers even if they were not 
in the program. 

At the time the data for this report were collected, most students in the study sample had 
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Exhibit 3 

Career Academies Evaluation 

Percentage of Students Who Indicated "High" Ratings on 
Measures of Interpersonal Supports, Motivational Processes, and Engagement, 
by Career Academy and Non- Academy Groups 
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SOURCE: MDRC calculations from the Career Academies Evaluation Student School Experience Questionnaire. 

NOTES: The measures shown above are summaries of students' ratings of several items from the Student School Experience Questionnaire. 

A two-tailed t-test was applied to differences between Career Academy and non-Academy groups. All differences are statistically significant 
at the 5 percent level or lower with the exception of the engagement measure, for which there was no difference between the two groups. 



little exposure to the integrated academic and occupation-related curricula, had not yet partici- 
pated in work-based internships, and had only a few opportunities to participate in activities that 
would prepare them for post-secondary education and careers. Thus, it is too early to tell whether 
the full Career Academy program will have a cumulative effect on student engagement and per- 
formance. 



How Do Career Academies Support Teachers? 

• Academy teachers indicate that they have more opportunities to collabo- 
rate with colleagues and to influence decisions in key areas of their work. 

The differences shown in the first three sets of bars in Exhibit 4 indicate the extent to 
which Career Academy teachers are exposed to a variety of institutional supports that are not as 
widely available in the regular high school environment. Resources include materials such as 
books and supplies, as well as nonmaterial resources including time and spaces to get together 
with colleagues. The teacher collaboration construct captures the degree to which teachers meet 
regularly to discuss instructional strategies, student-related issues, and curriculum integration. 
Areas of work over which teachers report a high degree of influence include instruction-related 
areas such as determining the content of professional development activities and selecting books, 
materials, and course content, and administrative areas such as disciplinary policies, elements of 
the daily schedule, and selecting students for their classes. Career Academy teachers were more 
likely than their non-Academy peers to give high ratings to each of these dimensions of institu- 
tional support. These variables are important because they are likely to be most sensitive to or- 
ganizational and policy changes reflected in the Career Academy approach. 

• Career Academy teachers were more likely than non-Academy teachers 
to indicate that they were part of a strong teacher learning community 
and that they emphasized personalized attention to their students. 

The fourth set of bars in Exhibit 4 shows that Academy teachers were more likely than 
their non-Academy counterparts to give a high rating on the indicators of a strong teacher learn- 
ing community. Key dimensions of a strong teacher learning community include whether teach- 
ers have opportunities to enhance subject matter knowledge and instructional strategies and to 
continue their professional growth. They also include indicators of whether teachers work closely 
with colleagues who are continually learning and seeking new ideas and who support their efforts 
to develop professionally. 

The fifth set of bars in Exhibit 4 indicates that Career Academy teachers were more likely 
than their non-Academy colleagues to place a high level of emphasis on personalized attention to 
students. This indicator includes the extent to which they try to be accessible to students, go out 
of their way to help them both academically and personally in school, and take an interest in stu- 
dents’ achievements and concerns beyond the classroom. 

« Career Academy teachers expressed higher levels of job satisfaction than 
non-Academy teachers. 



Exhibit 4 

Career Academies Evaluation 

Percentage of Career Academy and Non-Academy Teachers Who Indicated "High" Ratings 
on Measures of Institutional Supports, Interpersonal Supports, and Attitudes Toward Their Jobs 
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Academy teachers were more likely than non-Academy teachers to indicate a high level 
of satisfaction with their jobs (see Exhibit 4). For the purposes of this analysis, teachers satis- 
faction with their jobs includes the extent to which they were satisfied with the school learning 
environment, the intellectual challenge, and the enforcement of disciplinary policies. It also in- 
cludes teachers’ satisfaction with their work overall and whether they are likely to continue 
teaching. 

© Academy and non-Academy teachers were about equally likely to report 
a high level of effectiveness. 

As shown in Exhibit 4, about half of the Career Academy teachers indicated that they felt 
they were highly effective in their jobs, compared with 44 percent of the non-Academy teachers 
— not a statistically significant difference. This indicator is intended to capture teachers sense 
of the extent to which they believe that they can control or influence their students’ success in 
school and their sense of whether they are, in fact, making a difference in their students’ lives. 
Further analysis did not reveal any systematic differences between Academy and non-Academy 
teachers in the individual components of the self-reported effectiveness measure. 

In short, Academy teachers’ enhanced support, emphasis on personalized attention to stu- 
dents, and job satisfaction do not appear to have translated into systematic increases in their 
sense of effectiveness. One hypothesis for explaining this pattern of findings is that Career Acad- 
emy and non-Academy teachers may have differing definitions of “effectiveness.” In other 
words, Academy students may be performing better than non-Academy students, but the Acad- 
emy teachers may be expecting more from their Academy students and from themselves. The 
self-reported measure of teacher effectiveness used in this analysis may mask differences in other 
measures of effectiveness such as students’ assessments of their school experiences or assess- 
ments of student success indicated by school records. Also, the lack of difference in Academy 
and non-Academy teachers’ ratings of personal effectiveness should be interpreted cautiously 
because the measure does not take into account the possibility that Academy teachers may con- 
sider a greater number of performance dimensions as relevant. For example, interviews with Ca- 
reer Academy teachers revealed that many of them felt they should provide Academy students 
with a range of interpersonal and employability skills as well as academic skills. 

Finally, although the enhanced supports and satisfaction that Academy teachers experi- 
ence may be necessary conditions for greater effectiveness, they may not be sufficient. Academy 
teachers highlighted several significant challenges that must be met to attain the ideals repre- 
sented by the Career Academy model. For example, many teachers focused on the difficulty of 
integrating a rigorous vocational curriculum with a relevant and demanding academic curriculum 
and ensuring that the Academy’s work-based learning component provides students with expo- 
sure to the real world and a rich learning experience. 



Exploring Further Implications of the Findings 

Findings from the first two reports from the Career Academies Evaluation suggest that 
the approach holds some promise for restructuring high schools with the aim of creating more 
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supportive teaching and learning communities for students and teachers. Each of the sites in the 
study has demonstrated the feasibility of implementing and sustaining the core structural ele- 
ments of the Career Academy approach and adapting them to their local needs and circum- 
stances. The first report highlights some of the challenges involved in undertaking this 
restructuring. The current report indicates that, within the confines of these new structural ar- 
rangements, Academy students and teachers experience a greater degree of institutional and in- 
terpersonal support than do their peers in the regular high school environments. This enhanced 
support appears to have translated into somewhat higher levels of motivation among Academy 
students and higher levels of job satisfaction among Academy teachers. However, while students 
in their initial years in the Academy programs appear to be highly engaged in school, they are no 
more engaged than their non- Academy peers. 

While supports discussed in this report may provide some of the necessary conditions for 
improving students’ performance in high school and guiding them toward clearer pathways to 
post-secondary education and careers, it is still too early to tell whether the overall Career Acad- 
emy experience is sufficient to accomplish these goals. At this early stage in the evaluation, very 
few of the students had participated in the Academies’ work-based learning activities (which 
usually occur during or just after the 1 lth-grade year), and students’ exposure to the occupational 
theme and integrated curricula was still relatively limited. Also, as the types of supports noted 
earlier continue over two or three years, they could have a cumulative effect on student engage- 
ment and other outcomes such as progress toward graduation and advancement to higher-level 
academic and occupation-related courses. As a result, it is critical to follow students further 
through their high school years and beyond. Future reports from the evaluation will examine stu- 
dents’ exposure to the Career Academies’ integrated curricula and work-based learning activities 
and will include longer follow-up periods to examine the potential cumulative effects the 
Academies may have on students. Specifically, future reports will provide findings on whether 
the Career Academies are producing impacts on additional measures of student engagement in 
high school, their progress toward graduation, and their transition to post-secondary education 
and work. 

It is also important to note that these findings have been aggregated across all the sites 
and all students. These aggregated findings may mask differences among the sites that are asso- 
ciated with particular strategies for utilizing the basic elements of the Career Academy approach 
to support students and enhance their engagement in school. They may also hide differences 
among the participating high schools and school districts that could enhance or limit the supports 
available to Academy or non-Academy students. Future reports from this evaluation will explore 
variation across sites on a number of measures of program and contextual differences and their 
potential effects on students. 

Finally, these aggregated findings may obscure the fact that some subgroups of students 
may benefit more (or less) from the Career Academy experience than others. Future reports will 
also examine the effects Career Academies have on subgroups of students who are defined by 
background characteristics that, for example, are associated with a risk of poor performance in 
high school or that indicate a high level of prior school engagement. 



Chapter 1 

Introduction 



In response to today’s changing global economy, the national school-to-work (or school- 
to-career) movement has ushered in a variety of approaches to restructuring high schools in the 
United States. At stake is an attempt to support students academically while providing them with 
marketable skills and clearer pathways to a productive life beyond high school. Career Acade- 
mies, which are among the best established and most promising of these approaches, embrace the 
key principles of the school-to-work movement by integrating academic and vocational instruc- 
tion, providing work-based learning opportunities for students, and preparing students for post- 
secondary education, employment, or a combination of both. The Academies also reflect key 
principles of broader school reform initiatives by reconfiguring high schools into smaller, more 
personalized schools (the “school-within-a-school”), providing teachers with more control over 
their work through decentralized management, and developing interdisciplinary curricula. 

In the school-within-a-school organization that characterizes Career Academies, groups 
of students (usually 30 to 60 per grade in grades 9 through 12 or 10 through 12) take several 
classes together each year, in their regular high school, with the same group of teachers. The 
Academies focus on a career theme — such as health, business and finance, or electronics 
which is usually determined by local employment opportunities and evidence of growing de- 
mand for such expertise in the marketplace. Career Academies’ curricula consist of traditional 
academic classes (such as math, English, science, and social studies) combined with occupation- 
related classes that have a specific career theme. Teachers make an effort to integrate the aca- 
demic and occupation-related content and skills in their classes. Local employers from that field 
help plan and guide the program, and they serve as mentors and provide work experience for the 
students. For example, students may work as interns for a local employer as part of their gradua- 
tion requirements. 

The Career Academy approach was first developed in the late 1960s in Philadelphia as a 
strategy to prevent students from dropping out of high school and to help them prepare to enter 
the work force after graduation. By the mid-1990s, over 500 Career Academies had been estab- 
lished across the country through a variety of national, state, and school district initiatives. The 
goals of many Career Academies have also expanded to include improving all students’ engage- 
ment (that is, active and interested involvement) and performance in school and preparing them 
for post-secondary education as well as a career. That is, today’s Career Academies embrace a 
broad cross-section of high school students — not just those believed to be at risk of dropping 
out of high school. 

This report is the second in a series on an evaluation of High School Career Academies 
being conducted by the Manpower Demonstration Research Corporation (MDRC). The primary 
goal of MDRC’s Career Academies Evaluation is to measure the extent to which the Career 
Academies improve students’ engagement and performance in high school, their progress toward 
graduation, and their transitions to post-secondary education and careers. A total of 1,952 stu- 
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dents from 10 sites have entered the study sample over three school years from 1993-94 to 1995- 
96. MDRC will follow the students in the study sample through their scheduled graduation from 
high school. Eventually, the study may follow students through several years after their scheduled 
graduation from high school to learn about their enrollment and progress through post-secondary 
education, their employment and earnings, and other outcomes. MDRC will also examine the im- 
plementation of core components of the Academy approach in the 10 sites and explore how the ex- 
periences of Academy students and teachers differ from those of students and teachers in regular 
high school programs. The study is being conducted with support from the U.S. Departments of 
Education and Labor, 15 private funders, and the 10 participating sites. 

The first report from the evaluation was concerned primarily with describing the structural 
elements of the Career Academy approach as it had been implemented and sustained in the 10 
sites. 1 The current report begins to look inside the Academies to shed light on the experiences of 
their students and teachers and to contrast these experiences with those of their peers in the partici- 
pating high schools. In particular, this report focuses on the question of whether Career Academies 
function as “communities of support” for students and teachers. For students, such support includes 
a high level of involvement with and expectations from their teachers, opportunities to collaborate 
with other students who are engaged in school, enriched classroom instruction and work-related 
learning opportunities, and activities that prepare them for post-secondary education and careers. 
These dimensions of support have been identified in other research as important factors that in- 
crease motivation and engagement in high school and enhance students’ sense of how their work in 
high school is related to their future. 2 (Engagement includes students’ self-reported behavior, emo- 
tional state when in school, and psychological commitment to doing well in school.) For teachers, a 
community of support includes opportunities to collaborate with colleagues, adequate resources, 
influence over instructional and administrative decisions, and opportunities to develop personalized 
relationships with students. These factors have been found to be associated with increased job satis- 
faction and a greater sense of effectiveness among teachers. 3 

The goal of this report is to determine the extent to which students and teachers in the Ca- 
reer Academies experience such supports at higher levels than do students and teachers with similar 
characteristics in the regular high schools where the Academies are located. The report also exam- 
ines whether such differences, if they exist, are associated with higher levels of teachers’ self- 
reported sense of effectiveness and students’ self-reported motivation and level of engagement. 

The findings discussed here are preliminary in the sense that they focus on students’ ex- 
periences at a relatively early point in their involvement with the Career Academies or the regu- 
lar high school environments. Most students in the study sample were in the ninth or tenth grade 
at the time data were collected for this report, and a subsample had reached the eleventh grade. 
As a result, most students’ exposure to the Career Academies consisted primarily of enrollment 
in the school-within-a-school environment and its academic and occupation-related courses for 



'Kemple and Rock, 1996. 

2 Connell et al, 1995; Connell and Wellborn, 1991; Crichlow and Vito, 1989; Wehlage et al., 1989. 

3 Dembo and Gibson, 1985; Bandura, 1977, 1986. 
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one school year. At the time the data were collected, only a subset of the students in the study 
sample had been exposed to the Academies’ integrated curriculum and employer-provided ac- 
tivities for more than one school year, and very few of the students had participated in the 
Academies’ work-based learning activities. A longer follow-up period is needed to leam specifi- 
cally how — if at all — these activities improve students’ education in the Academies, including 
both the support they receive and the content of their courses (over and above what is available in 
regular high school environments), and to determine whether they enhance students’ engagement 
with and performance in school. 

Future reports from MDRC’s Career Academies Evaluation will provide findings on 
whether the Career Academies are producing impacts on additional measures of student engage- 
ment in high school, their progress toward graduation, and their transition to post-secondary edu- 
cation and work. These reports will also examine the relationships between program 
effectiveness and the types of process and implementation measures discussed in the current re- 
port. Finally, the evaluation includes a more intensive qualitative study covering factors both in 
and outside school that affect students’ progress toward high school graduation and post- 
secondary education and employment, and how these factors interact with the opportunities pro- 
vided by the Career Academies. 

The next two sections of this chapter summarize some of the primary problems that have 
been identified in large, comprehensive high schools, 4 and highlight the ways Career Academies 
are intended to address these problems. The remainder of the chapter describes the research de- 
sign that this evaluation is using to determine whether Career Academy students fare better then 
they would have if they had not had the opportunity to enroll in the programs. The final section 
provides an overview of the current status of the evaluation and summarizes the key findings 
from the first report. 



I. The Importance of Supporting Students and Teachers in High Schools 

Since the late 1970s, high schools have come to play an increasingly critical role in pre- 
paring young people to make two crucial and related transitions into the adult world: from de- 
pendence on parents and guardians to independence, and from schooling to work. On the one 
hand, high schools have attempted to facilitate students’ transitions to independence by providing a 
relatively secure environment in which adolescents can mature intellectually, socially, physically, 
politically, psychologically, and emotionally and by serving as the locus of a broad range of social 
services (such as drug counseling, health and family planning services, and child welfare services). 
On the other hand, as the global economy has placed a greater premium on higher levels of educa- 
tion, high schools have attempted to facilitate the transition from school to work by creating path- 
ways for students to attain the skills and other prerequisites needed for pursuing further education 
and a career. As a growing number of studies of high schools indicate, however, several structural 



4 Comprehensive high schools offer mostly academic and some vocational classes, as opposed to vocational high 
schools, to which students from area high schools travel to take vocational classes. 





features of many high schools have undermined their capacity to help students successfully make 
these transitions. At the same time, the research has revealed a number of reform strategies that 
have substantial promise for addressing these structural problems. 

A. Structural Problems in High Schools 

One theme identified across the board in research on the problems students face is the size 
of comprehensive high schools, particularly in cities. Many studies indicate that the sheer size and 
comprehensiveness of urban high schools often serve to depersonalize the school environment and 
prevent teachers from working as teams and developing an atmosphere conducive to learning . 5 In 
many large high schools, teachers rarely share the same group of students with a small group of 
colleagues, and students often have different classmates in each course. Students do not have a con- 
sistent group of teachers who are accountable for their success, and teachers do not have a chance to 
coordinate their course work with other teachers. Many students feel anonymous and isolated be- 
cause they know and are known well by only a small proportion of their classmates. In short, for 
many students this flux in classroom composition and student and teacher peer groups promotes 
feelings of alienation and diminishes any sense of community. 

Another common problem is the tracked curriculum, which is usually separated into higher- 
level academic classes for college-bound students and lower-level academic and vocational classes 
for those presumed not to be college-bound. For the academic disciplines, this separation often con- 
fines the teaching and learning process to the transfer of abstract knowledge from teachers to stu- 
dents or, in the case of vocational classes, to a narrow focus on specific job skills. Such classes 
rarely have opportunities to help students understand how basic skills, such as those learned in math 
or English, are applied outside the classroom . 6 

A third common problem is that high schools are isolated from other institutions, particu- 
larly employers, which often insulates students from the world of work instead of providing them 
with meaningful exposure to it . 7 With few connections among classes or between school and work, 
many students are inadequately informed about or are unprepared for post-secondary education and 
employment opportunities. 

To varying degrees, these problematic structural features are present in many high schools; 
however, they are much more prevalent in urban school districts that serve large numbers of low- 
income students, students of color, and students with limited English proficiency . 8 Students in these 
high schools are more likely to drop out; if they do graduate, they often lack the necessary skills 
and course work to attend college and start on the pathway toward high-wage jobs . 9 These struc- 



Sizer, 1984; Hill, Foster, and Gendler, 1990; Powell, Cohen, and Farrar, 1985; Felner, Primavera, and Cauce, 
1981; Roderick, 1993. 

6 Resnick, 1987a; Raizen, 1989; Stasz et al., 1993; Grubb, 1995. 

7 Resnick, 1987b; Berryman and Bailey, 1992; Berryman, 1995. 

8 Wehlage et al., 1989; Louis and Miles, 1990. 

^atriello, 1987; William T. Grant Foundation Commission on Work, Family, and Citizenship, 1988- Roderick 
1993. 
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tural features within the schools also parallel and perpetuate socioeconomic, racial, and ethnic dif- 
ferences in education and employment outcomes for students. 

The consequences of these problems in high schools are profound, in part, because suc- 
ceeding in high school has become more and more a necessary condition (though less and less a 
sufficient condition) of success in the labor market. At a minimum, a high school diploma sends a 
concrete signal to the adult world that an adolescent has learned valued skills and behaviors, such as 
consistent attendance, completing a set of required tasks, and cooperating with other people. High 
school graduation represents the achievement of competencies in key skills, including the use of 
texts, numbers, and other information needed for contemporary citizenship. The diploma, along 
with an adequate level of performance, is also a prerequisite for college and the better-paid and 
higher-status occupations. At the same time, the value of a high school diploma, without additional 
education or training, has declined dramatically in recent years. 10 High schools increasingly face the 
challenge of providing students with what have been called the “new basic skills,” which include 
the ability to solve problems, communicate effectively, work productively in groups, and use com- 
puters in addition to ensuring that students are able to read and do math. 11 Beyond these skills, 
youth development researchers and policymakers have argued that healthy and well-adjusted young 
people have positive and realistic perceptions of themselves and others, and can participate produc- 
tively in both one-on-one and group relationships and activities. 12 

B. Emphasizing Interpersonal Supports in High Schools 

Since the late 1980s, education researchers and policymakers have focused increasingly on 
restructuring high schools to support constructive relationships between students and teachers, and 
to develop teaching and learning strategies that integrate academic and occupation-related knowl- 
edge and skills. The goal of these reforms has been to increase students’ engagement in high school 
and to make them more prepared for further education and work. 

A series of studies on dropout prevention programs and schools have highlighted the im- 
portance of “school membership” in creating a supportive school community that can potentially 
“hold” students who might otherwise drop out. The studies define school membership as the proc- 
ess by which students seek “to belong and to be accepted as part of a peer group” and at the same 
time to have “the support and approval of adults.” 13 Through interviews with students in dropout 
prevention programs, these researchers found that students consistently described their new schools 
as friendlier and more caring places than previous schools in which they were failing and on the 
verge of dropping out. These students talked about peers who accepted them and teachers who 
cared about them and displayed a willingness to help them overcome both academic and personal 
problems. The researchers also found that a strong sense of school membership was associated with 
students’ willingness to reciprocate by participating in school activities and engaging more actively 
in both the academic and social life of the school. 



l0 See Mumane and Levy, 1996. 

"See Mumane and Levy, 1996, p. 9. 

l2 Pittman and Cahill, 1991; Zeldin, 1995a; Zeldin, 1995b; Pittman, Cahill, and Zeldin, 1994. 
"Wehlage et al., 1989, p. 114. 
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The findings from this research point out that these experiences contrasted sharply with the 
students’ experiences in large comprehensive high schools. Many of the students in the dropout 
prevention programs who were becoming engaged with their new schools had been demoralized by 
the impersonal adult and peer relationships in the high schools they had attended earlier, and had 
been overwhelmed by the alienation they felt when they could not find a niche where they could 
establish an identity. The result was a consistent pattern of course failures, absenteeism, and other 
school problems. Examining qualitative and quantitative data, these researchers found strong rela- 
tionships between the strength of students’ relationships with peers, teachers, and school and a vari- 
ety of measures of school engagement and performance. They concluded that, “by establishing a 
climate of trust and support, successful programs for at-risk youth help diminish isolation and en- 
hance self-esteem. Together these factors allow students to focus ... on the relationship between 
success in school and the possibility of a better future .” 14 

The importance of supportive school communities does not pertain solely to dropout 
prevention programs. The term “focus school” has been used to characterize private schools and 
special-purpose public schools, such as magnet schools and schools with a specific focus on the 
arts, humanities, or sciences, that develop clear missions to provide students with specific 
experiences and outcomes. Such schools typically have sufficient independence from 
administrative bureaucracies to initiate action in pursuit of their mission and to solve their own 
problems . 15 Like supportive school communities, a key feature of focus schools is their effort to 
communicate the reciprocal responsibilities of adults and students in the life of the school and to 
identify the benefits that each can derive from this pursuit. Research has also found that focus 
schools have a strong commitment to aggressively mold student attitudes and values and to 
promote a strong sense of community and caring. 

The structure of large comprehensive high schools has also been identified as a potential 
impediment to a supportive school environment because of its impact on teacher effectiveness. By 
restricting teacher autonomy and flexibility in adapting their instructional strategies, high schools 
often limit opportunities for professional growth and development, which in turn deprives students 
of optimal learning experiences. Researchers have pointed to several factors within large high 
schools that contribute to this problem, including few chances for teachers to meet and work to- 
gether, minimal influence over their work and the school environment, and a lack of material re- 
sources. It is important to note that efforts to improve high schools may well depend on such 
institutional supports as providing opportunities for teachers’ professional development . 17 

Recent research by Milbrey McLaughlin and Joan Talbert at Stanford University highlights 
the importance of “teacher learning communities” for stimulating and sustaining teachers’ learning 
and growth . 18 Teacher learning communities consist of groups of teachers who collaborate with 
each other within organizational boundaries, such as schools or subject area departments within 



l4 Wehlage et al., 1989, p. 174. 
l5 Hill, Foster, and Gendler, 1990, p. vii. 

16 Johnson, 1990; McLaughlin, 1993. 

17 Sarason, 1990. 

l8 Talbert and McLaughlin, 1994; McLaughlin and Talbert, 1993. 
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schools. These communities can promote greater teacher professionalism by encouraging the de- 
velopment of shared standards for curriculum and instructional strategies, norms for relationships 
with students and colleagues, and conceptions of good teaching practice. McLaughlin and Talbert 
call for policies that create more opportunities for teachers to engage in these communities . 19 
Teacher professionalism and learning opportunities have also been closely identified as being cen- 
tral to the creation of schools that function as communities of support for students . 20 



II. Career Academies as Communities of Support for Students and Teachers 

The research mentioned above identifies several common structural features of effective 
high schools that enhance the level of interpersonal and institutional supports available to students 
and teachers. Many of these features are reflected in the Career Academy approach. The school- 
within-a-school organization, integrated academic/occupational curricula, and employer partner- 
ships that form the cornerstones of the Career Academy approach provide direct responses to the 
common structural problems of high schools described above. In so doing, they offer unique op- 
portunities to help high school students make the transitions to independence, higher education, 
and productive work lives. 

Previous research on Career Academies echoes the themes that have been developed in the 
research on promising strategies for building more supportive and focused high school environ- 
ments. A review of early research on Academies highlighted a range of key factors that make the 
Academies distinctive from regular high school environments and contribute to their success . 21 One 
of the most frequent observations made both in these early studies and in subsequent research by 
others is that Academies are “like families .” 22 The Academies’ strong sense of community has been 
identified as one of their most powerful features. These studies concluded that Academies promote 
a supportive school climate where students and teachers know each other well and have high ex- 
pectations of each other. The Academies’ school-within-a-school organization creates a close-knit 
and caring community in which students and teachers form personalized bonds with one another. It 
allows teachers to give students individualized attention and to focus on both personal and aca- 
demic development. 

Another common theme found in previous studies of Career Academies is the notion that 
Academies function as professional support communities for teachers . 23 Academies provide teach- 
ers with more control over key decision-making activities that affect classroom and instructional 
practice. In addition, this research indicates that Academies provide a more supportive environment 
for teachers to collaborate with one another, share materials, and pursue innovative instructional 
strategies. Much of the Academy concept is built on strong teamwork designed to promote both 
better relationships among the faculty and higher standards for professional practice. 



l9 McLaughlin and Talbert, 1993. 

20 Newmann and Wehlage, 1995; McLaughlin and Talbert, 1993; Louis, Marks, and Kruse, 1995. 
2l Academy for Educational Development, 1989. 

22 See Academy for Educational Development, 1989, p. 16; Stem, Raby, and Dayton, 1992 , p. 95. 
23 Academy for Educational Development, 1989; Stem, Raby, and Dayton, 1992. 
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Figure 1.1 is a simplified conceptual framework that delineates the theory underlying the 
Career Academy approach and illustrates the pathways through which it is intended to enhance 
students’ success in high school and their transition to post-secondary education and careers. 24 The 
first column of Figure 1.1 shows the three key structural elements of the Academy approach: the 
school-within-a-school organization, an integrated academic/occupational curriculum, and em- 
ployer partnerships (to provide work experience). These structural elements should be viewed as 
institutional mechanisms designed to facilitate changes in the instructional and learning opportu- 
nities available to teachers and students. The second column of Figure 1.1 lists three types of 
supports that the Academies’ structural elements are designed to influence: (1) enhanced inter- 
personal support through the intensive interaction and collaboration offered by the school-within- 
a-school, (2) alternative methods of teaching and learning through the integration of academic 
and occupational curricula, and (3) work-based education through the employer partnerships. As 
shown in the figure, these supports are mutually reinforcing and, together, they are intended to 
affect students’ attitudes about, engagement with, and performance in high school (third column 
of Figure 1.1). 

Youth development and educational research suggest that schools-within-schools, the 
first structural component of Career Academies, should help strengthen the classroom environ- 
ment and thus improve students’ engagement in school. In addition, keeping smaller groups of 
classmates and teachers together for three or four years should provide the basis for building 
more stable and supportive relationships between and among the two groups. The second com- 
ponent, changes in school curricula to combine both academic and occupational instruction, is 
consistent with recommendations from the National Assessment of Educational Progress that in- 
clude “increasing the use of ‘hands-on’ examples and placing more problems in real-world con- 
texts to help students construct useful meanings for abstract concepts.” 25 The third component, 
the Academies’ work-based education through employer partnerships, is designed to foster stu- 
dents’ sense of how new skills can help them gain productive employment by exposing them to 
the work world and providing them with mentors from the workplace. 

Finally, the Academies seek to increase the rates at which students graduate from high 
school and to provide them with the necessary credentials and strategies to enter post-secondary 
education and employment (fourth and fifth columns of Figure 1.1). As shown in the figure, 
however, graduating from high school and acquiring various credentials should be viewed as 
transitional outcomes — as indications of the extent to which students are prepared for future 
education and work after high school. Ultimately, the Career Academies are intended to lead to 
higher levels of post-secondary education and to higher-skilled and higher-paying careers. 



24 An important goal of MDRC’s ongoing study of Career Academies is to document and measure the extent to 
which Academies’ structural changes improve students’ academic, behavioral, and work-related outcomes. The 
conceptual framework illustrated in Figure 1.1 has been used to guide MDRC’s selection of sites for the study, its 
specification of the research design, and its development of data collection instruments and analysis strategies. Chapters 
3 and 4 in this report highlight specific dimensions of this conceptual framework that were used to guide the analyses 
for examining students’ and teachers’ experiences, in the Academies and regular high school environment. 

25 Mullis, Owen, and Phillips, 1990. 
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Figure 1.1 

Career Academies Evaluation 
Simplified Model of the Career Academy Approach 
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Of course, the structural elements, teaching and learning opportunities, performance and 
adjustment indicators, and outcomes that are associated with the Career Academy approach are 
affected by the context in which they exist, including (1) state, district, and school policies and 
administrative practices; (2) characteristics of the local labor market, employers, and post- 
secondary education systems; (3) characteristics of students, families, teachers, and the local 
community; and (4) high school organization and curricula. 

In sum, Career Academies reflect the key principles of several key high school reform ef- 
forts aimed at helping students succeed in high school and prepare for further education and work. 
They represent the cornerstones of the school-to-work movement by integrating academic and vo- 
cational instruction, providing work-based learning opportunities for students, and preparing stu- 
dents for post-secondary education, employment, or a combination of both. The Academies also 
reflect key principles of broader school reform initiatives by restructuring high schools into smaller, 
more personalized schools, providing teachers with more control over their work through decen- 
tralized management, and engaging in interdisciplinary curriculum development. While Career 
Academies have existed since around 1970, they have undergone an extraordinary growth in their 
expansion across the country during the 1990s. The research being conducted for this study will 
shed light on the implementation and impact of this growing reform movement. 



III. The Career Academies Evaluation 

In 1993, the Manpower Demonstration Research Corporation (MDRC) began development 
work for a unique study of the Career Academy approach. Its primary purpose is to provide rigor- 
ous evidence about the efficacy of the theories and hypotheses embedded in the conceptual frame- 
work illustrated in Figure 1.1. The evaluation responds to the growing need for reliable information 
about the effectiveness of school-to-work and other major school reform initiatives by providing 
policy- and practice-relevant information on two broad questions: 

• To what extent do the Career Academies improve students’ school- and 
career-related performances over and above what they would have achieved if 
they had not had the opportunity to participate in an Academy? 

• How are the Career Academies different from the high school environments in 
which Academy students would otherwise have been enrolled, and how do 
these differences shape students’ post-secondary education and career prepa- 
ration? 

The Career Academies Evaluation consists of two components: the impact study, which 
addresses the first question above, and the process and implementation study, which addresses 
the second. When the evaluation is completed, these two components will be integrated to ex- 
plore the factors that help explain the Academies’ effectiveness or lack of effectiveness. 
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A. The Impact Study 



The primary focus of the impact study is to determine the extent to which the Career 
Academy approach improves students’ motivation and engagement in school, their progress 
toward graduation, and their preparation for and transitions to post-secondary education and 
work. Most studies, including the previous studies of Career Academies, aim to determine the 
effectiveness of a particular education intervention by comparing outcomes for students who are 
exposed to the intervention with outcomes for students who are not exposed to it. The primary 
challenge for such research is to minimize any differences between the two groups of students that 
were not caused by the intervention. To do this, it is necessary to ensure that students who are 
exposed to the intervention are as similar as possible (in terms of their background characteristics at 
the time they enter the intervention) to those students who are not exposed to the intervention. 

Several expert panels have noted that interventions like the Career Academies are especially 
challenging to evaluate because they target specific groups and, consequently, their recruitment and 
intake processes result in “selection bias” — that is, a program’s participants are likely to differ 
from people chosen for a comparison group in their motivation, prior performance in school, and 
other (unmeasurable) characteristics. 26 This difference is important because, for example, if a pro- 
gram targets the highest-achieving students in a school, in all likelihood that program will appear to 
be very effective when compared with the rest of the high school, which will include a high pro- 
portion of students in the low and middle ranges of achievement. Conversely, if a program targets 
the lowest-achieving students, it may appear to be ineffective when compared with the rest of the 
high school, which will include students in the high and middle ranges of achievement. The review 
panels concluded that unless a randomly assigned control group is used to provide a benchmark for 
the program group’s outcomes, it is typically impossible to determine the effectiveness of such in- 
terventions. 

The Career Academies Evaluation is extremely rare in the field of education research in 
that it has demonstrated the feasibility of implementing a random assignment research design 
within an ongoing high school program. This approach required that certain threshold conditions 
be present or created in each of the participating sites. 27 First, key stakeholders — including dis- 
trict administrators, teachers, parents, and students — had to agree that, if the Academies had 
more eligible applicants than they could serve, random assignment was a fair way to determine 
which applicants would be invited to participate. Toward that end, each of the Career Academies 
identified and recruited large numbers of eligible students for the study, resulting in nearly twice 
as many students applying for the Academies as the programs were able to serve. The Academies 
also had to modify their application process to accommodate two important requirements of the 
research design: (1) informing students and their parents about the study and gaining their con- 
sent to participate, and (2) having all applicants complete a questionnaire on their background 
characteristics and prior experiences in school. When each of these conditions was met, it was 



“Betsey, Hollister, and Papageorgiou, 1985; Job Training Partnership Act (JTPA) Advisory Committee, 1989; 
Atkinson and Jackson, 1992. 

27 See Chapter 4 in Kemple and Rock, 1996, for a more detailed description of how the random assignment 
procedure was implemented for this study. 



then feasible and appropriate to use random assignment to create the program and control groups 
for the evaluation. 

For this evaluation, a total of 1,952 students from 10 sites entered the research sample 
over three school years. All these students were determined by the respective Career Academies 
to be eligible and appropriate for participation in their programs. Of these, 1,063 students were 
randomly assigned to the program group and were admitted to the Academies. 28 The remaining 
889 students were randomly assigned to the control group, were not invited to participate in the 
Academies, and could choose other options in the high school or school district. In most cases, 
control group students enrolled in the general programs in the participating high school, but in 
some cases control group students enrolled in city-wide magnet programs or schools, or in voca- 
tional education programs. 29 

B. The Process and Implementation Study 

This portion of the study will document the key differences between the Career Acade- 
mies and the high schools within which they are located and where most of the control group 
students in the research sample are enrolled. First, it will investigate how the core components of 
the Academy model were implemented and sustained in the 10 study sites. It will also seek to 
explain how the experiences of Academy students differ from those of students in the regular 
high school programs. Finally, data will be obtained to learn how contextual factors (such as 
school district policies and trends in the local labor market) influence the Academies’ operation 
and effectiveness. 

To address these issues, MDRC is collecting several types of data on the study sites. 
First, data are being collected during a series of field research visits to each of the sites. These 
visits provide MDRC researchers with the opportunity to interview Academy teachers and stu- 
dents, school and district administrators, and local employer partners. MDRC staff also observe 
classes and other program activities, such as student recruitment and special events. Extensive 
qualitative information was also collected during the site selection process and during visits to 
the sites to monitor implementation of the research procedures. This information will be used to 
describe the particular characteristics of the participating Career Academies and their local con- 
texts. 



Survey data are also being collected and will be used to make systematic comparisons 
between the experiences and perceptions of Academy students and teachers and those of their 
non- Academy counterparts. Finally, MDRC is collecting data from student transcripts and school 



28 In order to ensure that the Career Academy programs were able to operate at capacity during the period of the 
evaluation, a total of 1,247 students were randomly selected for admission to the programs. The 1,063 students in the 
study’s program group are a randomly selected subset of all the students randomly assigned to the Academies. In 
addition, 157 students were randomly assigned to waiting lists for the various Academies before students were selected 
for the control group, in the event that the programs experienced higher rates of attrition than expected. 

9 Magnet schools or programs are racially mixed public schools usually established to meet desegregation goals, 
which draw students from throughout a school district on a voluntary basis and offer an innovative educational 
program. 
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administrative records to document the students’ patterns of enrollment and attendance in Acade- 
my programs while they are in high school. 

C. Sites Participating in the Career Academies Ev aluation 

The 10 sites participating in the Career Academies Evaluation were chosen strategically, 
with the goal of providing a credible test of the Career Academy approach as it had been defined in 
previous research and implemented in a broad range of settings. 30 MDRC sought to ensure that the 
selected Career Academies were well established rather than in the initial or partial stages of im- 
plementation. At the same time, it was important that the participating Academies not be 
“hothouse” programs — that is, incapable of being implemented under a broad range of conditions 
and circumstances. Thus, Academies were selected to include school districts and high schools re- 
flecting the diversity of settings (urban centers and small cities) under which Career Academies 
have been implemented. MDRC was specifically interested in Academies serving a broad range of 
students, including those who were perceived to be at risk of not succeeding in the regular high 
school environment. 

In addition, MDRC sought high schools in which there was a clear contrast between the Ca- 
reer Academy and other programs available to students. This was important because one of the 
primary questions for the study was whether the Career Academy approach improves student out- 
comes above and beyond what would have occurred had they not had the opportunity to attend an 
Academy. Some schools and school districts operate more than one Career Academy or other 
Academy-like programs, such as school- within-a-school theme programs, or school-to-work transi- 
tion programs that include integrated curricula and work-based learning experiences. If such 
schools were included in the study, a high proportion of students in both the program and control 
groups would probably have been enrolled in similar programs. This situation would reduce the 
contrast between students’ experiences in the program and control groups and could make the 
Academies appear ineffective. Finally, the key stakeholders in the school system — the school 
principal and other administrators, teachers, and school district officials — agreed to participate in 
the study and to cooperate with the requirements of the research design. 

Figure 1.2 shows the names, locations, and affiliations of the 10 Career Academies partici- 
pating in the evaluation. Most of the nine school districts in the evaluation (one district — San Jose, 
California — includes two of the participating Career Academies) are large and enroll substantial 
percentages of black and Hispanic students compared with national averages. The participating 
school districts also, on average, have higher dropout rates, unemployment rates, and percentages of 
low-income families. Most Career Academies across the country are located in such districts, and 
MDRC purposely sought such sites for the Career Academies Evaluation. 

All of the sites had established the basic Career Academy components mentioned above: a 
school-within-a-school organization, an integrated academic/occupational curriculum, and em- 
ployer partnerships. This combination of features was not available elsewhere in the participating 



30 See Chapter 2 in Kemple and Rock, 1996, for a more detailed description of the criteria and process used to select 
sites for this study. 
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Figure 1.2 

Career Academies Evaluation 

Names, Locations, and Affiliations of Participating Career Academies 




1 . 


Academy and High School 

Business and Finance Academy 
George Westinghouse High School 


School District and Citv 

Pittsburgh Public Schools 
Pittsburgh, Pennsylvania 


Academy Network and 
School Year Academy Started 
Independent 
1984-85 


2. 


Academy of Finance 

Lake Clifton/Eastern High School 


Baltimore City Public Schools 
Baltimore, Maryland 


National Academy Foundation 
1987-88 


3. 


Public Service Academy 
Anacostia High School 


District of Columbia Public Schools 
Washington, D.C. 


D.C. Public Schools Academy Network 
1989-90 


4. 


Academy for Aerospace Technology 
Cocoa High School 


Brevard County Public Schools 
Cocoa, Florida 


Florida's Academies for Career Development 

and Applied Technology 

1993-94 


5. 


Academy of Travel and Tourism 
Miami Beach Senior High School 


Dade County Public Schools 
Miami Beach, Florida 


National Academy Foundation 
1991-92 


6. 


Health Professions Academy 
Socorro High School 


Socorro Independent School District 
Socorro, Texas 


Independent 

1991-92 


7. 


Global Business Academy 
Valley High School 


Santa Ana Unified School District 
Santa Ana, California 


California Partnership Academy 
1991-92 


8. 


Watsonville Video Academy 
Watsonville High School 


Pajaro Valley Unified School District 
Watsonville, California 


California Partnership Academy 
1991-92 


9. 


Electronics Academy (SC) 
Silver Creek High School 


East Side Union High School District 
San Jose, California 


California Partnership Academy 
1984-85 


10. 

O 

ERIC 


Electronics Academy (I) 
Independence High School 


East Side Union High School District 
San Jose, California 

41 
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California Partnership Academy 
1984-85 



high schools. 31 Each Career Academy attempted to serve a wide range of students, including those 
who appeared to be at risk of dropping out. 

Figure 1 .2 indicates that the participating Academies offer a range of occupational themes: 
Three are in the business and finance fields; three focus on high-technology areas such as electron- 
ics and aerospace technology; and one each is in the fields of health occupations, public service, 
travel and tourism, and video technology. The participating programs were drawn from most of the 
major, established networks of Career Academies across the country, with four from the California 
Partnership Academy network, two from the National Academy Foundation network, one from the 
Florida network of Academies for Career Development and Applied Technology, and one from the 
network of Academy programs created by the District of Columbia Public Schools. Two of the 
participating Academies were developed independently through local high school or district initia- 
tives. Figure 1.2 also indicates that, as of the 1994-95 school year (when the last sites joined the 
study), the participating Career Academies had been in operation for as few as 2 years and as many 
as 10 years. 

In summary, the sites participating in the Career Academies Evaluation provide a solid 
foundation on which to build a credible assessment of the implementation and impact of the Career 
Academy approach. Two important cautions should be kept in mind, however, when interpreting 
the findings from this study. First, because the participating sites were chosen strategically, rather 
than randomly, the findings from this study cannot necessarily be generalized to all schools and- 
school districts that operate Career Academies. While the sites as a group share the characteristics 
of typical urban and small-city school districts and, individually, reflect much of the diversity of 
such districts, they may differ in important ways that limit the generalizability of the findings. Sec- 
ond, many of the reports that are produced from this study will focus on findings that are aggre- 
gated across the full sample of students and sites. While such findings shed light on the impact or 
implementation of the Career Academy approach more generally, they may mask important sources 
of variation among the sites. As the study proceeds, therefore, an effort will be made to determine 
whether particular sites or groups of sites provide useful lessons about the potential strengths and 
limitations of the Academy approach in particular contexts. Third, like their host high schools and 
school districts, the participating Career Academies are dynamic and evolving. They must confront 
factors such as staff turnover, increases or decreases in funding, changes in local or state education 
policy, shifting levels of support from building or district staff, and changes in the amount and 
types of support they receive from employer partners. Because this is a longitudinal study, it will be 
able to provide a realistic picture of how ongoing programs evolve and change in the context of dy- 
namic high schools. As a result, the reports from the study will highlight the periods reflected in the 
data and indicate key contextual factors that influence the applicability of the findings. 



3 'Although some of the participating high schools do operate programs that they classify as Career Academies, 
information collected for this study indicated that most of these programs do not include the core characteristics of the 
Academy approach described above. As a result, the participating Career Academy programs represent a clear contrast 
with the other program in the high schools. 
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IV. Xhe Current Status of the Career Academies Evaluation and an Overview of the 

Current Report 

By the end of the 1995-96 school year (the most recent period for which data are available), 
the students in the study s program and control groups had been participating for a minimum of one 
year and a maximum of three years. Data are being collected from students’ school records and 
from surveys administered to Academy and non-Academy students and teachers. MDRC is also 
collecting data through field research activities including individual and group interviews with key 
stakeholders and observations of school- and work-related activities. 

A. Highlights of the First Report 

The first report on the Career Academies Evaluation — Career Academies: Early Imple- 
mentation Lessons from a 10-Site Evaluation (1996) — described the basic foundation for the study 
and focused on information collected during the start-up phase and data collected through the 
1994-94 school year. It discussed the criteria and process used to select sites for the study and de- 
scribed the participating districts and Career Academies. The first report also described the student 
recruitment, application, and selection processes used by the Career Academies, and explained 
how the random assignment procedures were incorporated into existing application and selection 
procedures. The report examined the background characteristics of the students in the research 
sample and their families, and included a discussion of the extent to which these characteristics 
indicate that students in the research sample may be at risk of dropping out of high school. Fi- 
nally, the report offered some initial findings on students’ patterns of participation in the pro- 
grams, described the teachers in the Career Academies, and compared their perspectives on 
teaching and their work environment with those of their colleagues in the regular high school 
programs. 

Utilizing data collected from field research, interviews, and surveys, the 1996 report pre- 
sented several significant findings about the Career Academies in this evaluation: 

• All 10 of the participating high schools had implemented the Academy ap- 
proach’s structural elements: a school-within-a-school, a curriculum that com- 
bines academic and occupation-related courses oriented toward a career theme, 
and partnerships with local employers. This finding shows that the evaluation 
has the basis for a valid test of the effectiveness of the Career Academy ap- 
proach as it exists in a range of high schools. 

® The participating Career Academies encompass considerable variation in their 
numbers of students and teachers, numbers and content of Academy courses, 
types of opportunities for teacher collaboration, degree of curriculum integra- 
tion, and the role and scope of employer involvement. Such variation under- 
scores the adaptability of the Academy approach to each school’s needs and 
circumstances, and demonstrates that the approach can be implemented in a 
wide range of school settings. 
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• The participating Career Academies have attracted large numbers of applicants 
with a high degree of demographic and educational diversity. Their broad appeal 
extends to students who are at risk of performing poorly or of dropping out of 
school, as well as to students who do well in school. 

• A large majority (83 percent) of the students who were selected to participate in 
the Career Academies enrolled in them, and more than two-thirds of those who 
enrolled were still participating two years later. Given the high rate of school 
transfers among similar, non-Academy students, these rates of enrollment and 
retention should be viewed as substantial. 

In summary, an important conclusion from the first report was that the group of partici- 
pating sites and the student research sample provided the opportunity for a credible test of the 
Career Academy approach. They also provided the opportunity to learn about how the approach 
can be adapted to local circumstances and whether it is more effective under particular circum- 
stances or for particular groups of students. 

B. Overview of This Report 

While the first report from the evaluation was concerned primarily with describing the 
structural elements of the Career Academy approach as they have been implemented and sus- 
tained in the 10 sites, the current report begins to look inside the Academies to shed light on the 
experiences of their students and teachers and to contrast these experiences with those of their 
peers in the regular high school environments. The findings discussed in this report are based on 
the questionnaire data collected from Academy and non-Academy students and teachers in the par- 
ticipating sites. The report also draws on data collected from open-ended interviews with students, 
teachers, and administrators and from observations of Academy classes and activities. Chapter 2 
describes the data sources and research samples used in this report, and presents several analytical 
issues relevant to the interpretation of the findings. 

Chapter 3 examines the extent to which the Career Academy approach increases students 
self-reported exposure to selected dimensions of school-related support from teachers, peers, par- 
ents, and other adults. The school-related supports that are examined include personalized attention 
from teachers and teachers’ expectations of students in both academic and nonacademic areas, lev- 
els of engagement or detachment on the part of peers, collaboration among students, and involve- 
ment from parents and other adults in students’ lives. Prior research, as well as findings from the 
data collected for this study, indicate that students who experience higher levels of these supports 
are more likely to report intrinsic motivations for participating in school activities — that is, they 
are engaged in school because they enjoy it, not because of external pressure — and to see strong 
connections between what they are learning in school and their futures. Increases in student moti- 
vation are, in turn, related to higher levels of students’ self-reported engagement. A central question 
for Chapter 3, therefore, is whether any increase in the types of school-related supports provided by 
the Career Academies (over and above what students experience in regular high school environ- 
ments) leads to increases in students’ self-reported motivation and engagement in school. 
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Chapter 4 examines the extent to which the Career Academies provide teachers with sup- 
ports that enhance their satisfaction with their work and their sense of effectiveness in making a 
difference in their students’ lives. This analysis is important because the potential effectiveness 
of the Career Academy approach will depend, in large part, on the work teachers do to make use 
of its structural elements. In general, this work focuses on constructing a supportive learning en- 
vironment for students that includes a coherent and rigorous curriculum and opportunities to 
connect school- and work-based learning. Chapter 4 also explores whether the structural changes 
created by the Career Academy approach are associated with differences in teachers’ level of col- 
laboration with each other, in their classroom resources and control over their work, and in their 
relationships with students. 
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Chapter 2 

Data Sources, Research Samples, and Analytic Issues 



The analysis conducted for this report involves a comparison between the school-related 
experiences, perceptions, and behavior of students and teachers associated with the Career 
Academies and those of similar students and teachers in other parts of the participating high 
schools. This information provides early insights into the extent to which the Career Academies 
may be supporting students and teachers in ways that regular high school environments do not 
and in ways that may affect students’ engagement in school and teachers sense of effectiveness. 

To address these issues, the report relies on data collected from students and teachers in the 
participating high schools using the Career Academies Evaluation Student School Experience 
Questionnaire (SEQ) and the Career Academies Evaluation Teacher Questionnaire, respectively. 1 
The first section of this chapter presents the survey sampling strategy and response rates for the 
SEQ and examines the background characteristics of the Academy and non- Academy students who 
completed it. It concludes that there are no systematic differences in the background characteristics 
of these two groups of students. As a result, differences between Academy and non-Academy stu- 
dents’ school-related experiences, perceptions, and behavior based on measures developed from the 
SEQ can be attributed to differences between the Academy and non-Academy school environments 
in which the students were enrolled. 

The Teacher Questionnaire was completed by 65 Career Academy teachers and 403 non- 
Academy teachers in the 10 participating high schools. Because teachers are not assigned randomly 
to the Academies or to regular high school classes, an important goal of the Teacher Questionnaire 
was to determine whether Academy teachers differ substantially from their non-Academy col- 
leagues in terms of background characteristics, training, subject area specialties, and teaching expe- 
rience. The second section of this chapter discusses the sampling strategy and response rates for the 
Teacher Questionnaire and examines the background characteristics of the Academy and non- 
Academy teachers who completed it. It concludes that Academy and non-Academy teachers are 
similar in terms of a number of background characteristics, education levels, and average number of 
years of teaching experience. However, the Academy and non-Academy groups of teachers differ 
somewhat in their racial composition, distribution of teaching experience, and subject area special- 
ties. This finding suggests that differences that emerge based on teachers’ responses to the ques- 
tionnaire items may be due, in part, to differences in their characteristics as well as to differences in 
their Academy and non-Academy teaching environments. Analyses presented later in this report 
take this possibility into account by controlling statistically for the differences in measured charac- 
teristics. As a result, differences that emerge from these analyses between Academy and non- 
Academy teachers’ experiences, perceptions, and behavior based on measures developed from the 
Teacher Questionnaire are more likely to be attributable to the differences between the Career 
Academies and the regular school environments in which they work than to differences between the 
characteristics of the two groups. 



'Copies of these instruments can be obtained from the Manpower Demonstration Research Corporation. 



O 

ERIC 



- 19 - 



46 



